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Abstract 



We consider the problem of optimizing a nonlinear objective function over a weighted indepen- 
dence system presented by a linear-optimization oracle. We provide a polynomial-time algorithm 
that determines an r-best solution for nonlinear functions of the total weight of an independent set, 
where r is a constant that depends on certain Frobenius numbers of the individual weights and is 
independent of the size of the ground set. In contrast, we show that finding an optimal (0-best) 
solution requires exponential time even in a very special case of the problem. 

1 Introduction 

An independence system is a nonempty set of vectors S Q {0, 1}" with the property that x € {0, 1}" , 
X < y S implies x € 5" . The general nonlinear optimization problem over a multiply-weighted 
independence system is as follows. 

Nonlinear optimization over a multiply-weighted independence system. Given independence 
system S Q {0, 1}" , weight vectors if"*^, . . . , if*^ € Z" , and function / : Z'^ — > M , find x G 5 minimizing 
the objective 



The representation of the objective in the above composite form has several advantages. First, for 
d > 1 , it can naturally be interpreted as multi-criteria optimization: the d given weight vectors 
, . . . ,w'^ represent d different criteria, where the value of x G 5 under criterion i is its i-th total 



d given criteria by the given function / . Second, it allows us to classify nonlinear optimization 




weight w'^x = Yli 



w^jXj ; and the objective is to minimize the "balancing" f{w^x, . . . , 



w'^x) of the 



problems into a hierarchy of increasing generality and complexity: at the bottom lies standard linear 
optimization, recovered with d = 1 and / the identity on Z ; and at the top lies the problem of 
minimizing an arbitrary function, which is typically intractable, arising with d = n and Wi = Ij the 
z-th standard unit vector in Z*^ for all i . 

The computational complexity of the problem depends on the number d of weight vectors, on the 
weights Wj , on the type of function / and its presentation, and on the type of independence system 
S and its presentation. For example, when S is a matroid, the problem can be solved in polynomial 
time for any fixed d , any {0, 1, . . . ,p}-valued weights tt;* with p fixed, and any function / presented 
by a comparison oracle, even when S is presented by a mere membership oracle, see Also, when 
S consists of the matchings in a given bipartite graph G , the problem can be solved in polynomial 
time for any fixed d , any weights if* presented in unary, and any concave function / , see [3]; but on 
the other hand, for convex f , already with fixed d = 2 and {0, l}-valued weights Wj , it includes as a 
special case the notorious exact matching problem, the complexity of which is long open OE]. 

In view of the difficulty of the problem already for d = 2 , in this article we take a first step 
and concentrate on nonlinear optimization over a (singly) weighted independence system, that is, with 
d = 1 , single weight vector w = {wi, . . . , Wn) G Z" , and univariate function / : Z ^ M . The function 
/ can be arbitrary and is presented by a comparison oracle that, queried on y € Z , asserts whether 
or not f{x) < f{y) . The weights wj take on values in a p-tuple a = (oi, . . . ,ap) of positive integers. 
Without loss of generality we assume that a = (ai, . . . , Op) is primitive, by which we mean that the Oi 
are distinct positive integers having greatest common divisor gcd(a) := gcd(ai, . . . ,ap) that is equal 
to 1 . The independence system S is presented by a linear- optimization oracle that, queried on vector 
G Z" , returns an element x & S that maximizes the linear function vx = Y2]^=i ^j^j ■ turns 
out that solving this problem to optimality may require exponential time (see Theorem 17. ip . and so 
we settle for an approximate solution in the following sense, that is interesting in its own right. For 
a nonnegative integer r , we say that x* S S is an r-best solution to the optimization problem over 
S if there are at most r better objective values attained by feasible solutions. In particular, a 0-best 
solution is optimal. Recall that the Frobenius number of a primitive a is the largest integer F(a) that 
is not expressible as a nonnegative integer combination of the . We prove the following theorem. 

Theorem 1.1. For every primitive p-tuple a = (ai, . . . , Op) , there is a constant r{a) and an algorithm 
that, given any independence system S C {0, 1}" presented by a linear- optimization oracle, weight 
vector w € {oi, . . . , Op}" , and function / : Z — > M presented by a comparison oracle, provides an r{a)- 
best solution to the nonlinear problem mm{f{wx) : x & S} , in time polynomial in n . Moreover: 

1. If ai divides a,_|_i for i = \, . . . ,p — 1 , then the algorithm provides an optimal solution. 
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2. For p = 2 , that is, for a = (01,02) , the algorithm provide an F{a)-best solution. 

In fact, we give an explicit upper bound on r(a) in terms of the Frobenius numbers of certain 
subtuples derived from a . 

Because F(2,3) = 1 , Theorem 11.11 (Part 2) assures us that we can efficiently compute a 1-best 
solution in that case. It is natural to wonder then whether, in this case, an optimal (i.e., 0-best) 
solution can be calculated in polynomial time. The next result indicates that this cannot be done. 

Theorem 1.2. There is no polynomial time algorithm for computing an optimal (i.e., 0-best) solution 
of the nonlinear optimization problem inm{f(wx) : x E 5} over an independence system presented by 
a linear optimization oracle with f presented by a comparison oracle and weight vector w € {2,3}'^. 

The next sections gradually develop the various necessary ingredients used to establish our main 
results. J}2] sets some notation. J}3] discusses a naive solution strategy that does not directly lead to 
a good approximation, but is a basic building block that is refined and repeatedly used later on. ^ 
describes a way of partitioning an independence system into suitable pieces, on each of which a suitable 
refinement of the naive strategy will be applied separately. J|5] provides some properties of monoids and 
Frobenius numbers that will allows us to show that the refined naive strategy applied to each piece 
gives a good approximation within that piece, ^combines all ingredients developed in ^SHSl provides 
a bound on the approximation quality r(a) , and provides the algorithm establishing Theorem II. li 
^ demonstrates that finding an optimal solution is provably intractable, proving a refined version of 
Theorem 11.21 ^concludes with some final remarks and questions. 

2 Some Notation 

In this section we provide some notation that will be used throughout the article. Some more specific 
notation will be introduced in later sections. We denote by M , M+ , Z and Z_|_ , the reals, nonnegative 
reals, integers and nonnegative integers, respectively. For a positive integer n, we let := {1, . . . , n} . 
The j-th standard unit vector in M" is denoted by Ij . The support of x € M" is the index set 
supp(2;) := {j : Xj 7^ 0} C of nonzero entries of x . The indicator of a subset J C is the vector 
1 J := J Ij G {0, 1}" , so that supp(l j) = J . The positive and negative parts of a vector x € M" 
are denoted, respectively, by x~^,x~ € M" , and defined by xf := max{xj,0} and x^ := — min{xj,0} 
for z = 1, . . . , n . So, x = x~^ — x~ , and xfx^ = for i = 1, . . . , n . 

Unless otherwise specified, x denotes an element of {0, 1}" and X,fj,,T, v denote elements of . 
Throughout, a = (oi, . . . , Op) is a primitive p-tuple, by which we mean that the Oj are distinct positive 
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integers having greatest common divisor gcd(a) := gcd(ai, . . . , Op) equal to 1 . We will be working 
with weights taking values in a , that is, vectors w G {ai, . . . ,ap}^ . With such a weight vector w 
being clear from the context, we let Ni := {j N : wj = Oj} for i = 1, . . . ,p , so that N = (+Jf=;^ Ni . 
For X G {0, 1}"" we let Ai(x) := |supp(x) (iNi] for i = 1, . . . ,p , and A(x) := (Ai(x), . . . , Xp{x)) , so that 
wx = X{x)a . For integers z, s E Z and a set of integers Z C Z , we define z + sZ := {z + sx : x G Z} . 

3 A Naive Strategy 

Consider a set C {0, 1}" , weight vector w G {ai, . . . , Op}" , and function / : Z — > M presented by a 
comparison oracle. Define the image of S under w to be the set of values wx taken by elements of S , 



As explained in the introduction, for a nonnegative integer r , we say that x* G 5" is an r-best solution 
if there are at most r better objective values attained by feasible solutions. Formally, x* G S* is an 
r-best solution if 



We point out the following simple observation. 

Proposition 3.1. /// is given by a comparison oracle, then a necessary condition for any algorithm 
to find an r-best solution to the problem mm{f{wx) : x G S"} is that it computes all but at most r 
values of the image vu ■ S of S under w . 

Note that this necessary condition is also sufficient for computing the weight wx* of an r-best 
solution, but not for computing an actual r-best solution x* G 5 , which may be harder. 

Any point x attaining max{ii;x : x G S"} provides an approximation of the image given by 

(1) {wx : X < x} C w-S C {0, 1, . . . , fwx} . 

This suggests the following natural naive strategy for finding an approximate solution to the optimiza- 
tion problem over an independence system S that is presented by a linear-optimization oracle. 



input independence system S C {0, 1}" presented by a linear-optimization oracle, / : Z ^ M 
presented by a comparison oracle, and w G {ai, . . . , a^}" ; 

obtain x attaining max{u)x : x G S"} using the linear-optimization oracle for S ; 

output X* as one attaining mm{f(wx) : x < x} using the algorithm of Lemma 13.31 below . 



w ■ S 




\{f{wx) : f{wx) < f{wx*) , X G S}\ < r . 



Naive Strategy 
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Unfortunately, as the next example shows, the number of values of the image that are missing from the 
approximating set on the left-hand side of equation ([T|) cannot generally be bounded by any constant. 
So by Proposition I3.H this strategy cannot be used as is to obtain a provably good approximation. 

Example 3.2. Let a := (1, 2) , n := 4m , y := Yl'i=i Ij i := ^i=2m+i > ™^ w := y + 2z , that is, 

y = (1,...,1,0,...,0), z = (0,...,0,1,...,1), w = (1,...,1,2,...,2), 

define / on Z by 

f{k) 

and let S be the independence system 

5- := {xg{0, 1}" : x<y} U {x G {0, 1}" : x < z} . 

Then the unique optimal solution of the linear-objective problem max{'wx : x ^ S} is x := z , with 
wx = 4m , and therefore 

{wx : X < x} = {2i : i = 0,1, . . . , 2m} , and 

w S = {i : i = 0,1,... ,2m} U {2i : i = 0,1, . . . ,2m} . 

So all m odd values (i.e., 1,3,..., 2m — 1) in the image w ■ S are missing from the approximating set 
{wx : X < x} on the left-hand side of ([T|), and x* attaining mm{f{'wx) : x < x} output by the 
above strategy has objective value f{wx*) = 2m , while there are m = ^ better objective values (i.e., 
1,3,..., 2m — 1) attainable by feasible points (e.g., J2i=i 1* , for A: = 1, 3, ... , 2m — 1). 

Nonetheless, a more sophisticated refinement of the naive strategy, applied repeatedly to several 
suitably chosen subsets of S rather than S itself, will lead to a good approximation. In the next 
two sections, we develop the necessary ingredients that enable us to implement such a refinement of 
the nai've strategy and to prove a guarantee on the quality of the approximation it provides. Before 
proceeding to the next section, we note that the naive strategy can be efficiently implemented as 
follows. 

Lemma 3.3. For every fixed p-tuple a , there is a polynomial-time algorithm that, given univariate 
function / : Z — s- M presented by a comparison oracle, weight vector w € {ai,...,ap}" , and x € 
{0, 1}" , solves 

mm{f{wx) : x < x} . 




k , k odd; 
2m , k even. 
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Proof. Consider the following algorithm: 



input function / : Z — M presented by a comparison oracle, w G {oi, . . . , Cp}" and x G {0, 1}" ; 

let Ni := {j : wj = Cj} and Tj := Xi{x) = |supp(2;) H Ni\, i = 1, . . . ,p ; 

for every choice of = (i^i, . . . , i/p) < (ti, . . . ,Tp) = r do 

determine some Xy < x with \i{xp) = |supp(xjy) f\ Ni\ = Vi, i = 1, . . . ,p ; 

end 

output X* as one minimizing f{wx) among the Xi^ by using the comparison oracle of / . 

Since the value wx depends only on the cardinalities |supp(x) n A'^jl, i = 1, . . . ,p , it is clear that 

{wx : x < x} = {wxu : v <t} . 

Clearly, for each choice < r it is easy to determine some x^, < x by zeroing out suitable entries of 
X . The number of choices v < t and hence of loop iterations and comparison-oracle queries of / to 
determine x* is 

n(T, + l) < (n + lf. 

i=l 

□ 

4 Partitions of Independence Systems 

Define the face of S CI {0, 1}" determined by two disjoint subsets L,U N = {1, ... ,n} to he 

si := {x e S : Xj = for j G L , xj = I for j e U} . 
Our first simple lemma reduces linear optimization over faces of S to linear optimization over S . 

Lemma 4.1. Consider any nonempty set S C {0, 1}" , weight vector w Ci 1,"' , and disjoint subsets 
L,U C N . Let a := 1 + 2nmax lu^jl , let 1l,^u £ {0,1}"' be the indicators of L,U respectively, and 
let 

V := max {(w + a(l(7 — 1l))x : x G S} — \U\a 
(2) = max jwx - a (T.jeui'^ " ^i) + T^jeL^j) ■ x e . 

Then either v > — ^a, in which case max{wx : x G S^} = v and the set of maximizers of wx over 
si is equal to the set of maximizers of the program or v < —^a , in which case S^ is empty. 
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Proof. For all x G {0, 1}" , we have —^a < wx < , and so for all y £ S \ and z G we have 



wy — a 



i^i'^ - Vj) + J2yA - wy-a < ^a-a = - 



< wz = wz — a 



□ 



Let S C {0,1}" and w G {oi, . . . , ap}" be arbitrary, and let Ni := {j G N : wj = aj} as 
usual. As usual, for x G S" , let Xi{x) := |supp(x) fl Ni\ for each i . For p-tuples /i = (^i, . . . ,/ip) and 
A = (Ai, . . . , Ap) in Z!|_ with /u < A , define 

(3) S'^ := LeS : ^^(-)=^- if^.<A., 1 _ 

[ Ai(x) > Hi , if /Ui = Aj . J 

Proposition 4.2. Let S C {0, 1}" be arbitrary. Then every A G induces a partition of S given by 

S = 1+J . 

Proof. Consider any x £ S , and define ^ < A by /i, := min{Ai(x), Aj} . Then x £ , but x ^ for 
u < X , v ^ fj, . □ 

Lemma 4.3. For a// /ixed p-tuples a and A G , there is a polynomial-time algorithm that, given 
any independence system S presented by a linear- optimization oracle, w G {oi, . . . ,ap}" , and /i G 
with /U < A , solves 

max|t(;x : x € S^^ . 
Proof. Consider the following algorithm: 



input independence system S C {0, 1}" presented by a linear-optimization oracle , 
w G {ai, . . . , Op}" , and fi < X ; 

let I := {i : fii < Aj} and A'^; := {j £ N : Wj = ai}, i = 1, . . . ,p ; 
for every Si C Ni with \Si\ = fJ-i, i = 1, . . . ,p, if any, do 
let L := Uie, {N^ \ S,) and U := [j^i ; 

find by the algorithm of Lemma l4.ll an x{Si, . . . , Sp) attaining mao<.{wx : x G S^} if any; 
end 

output X* as one maximizing tDX among all of the x{Si, . . . , Sp) (if any) found in the loop 
above . 
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It is clear that is the union of the over all choices 5*1, . . . , 5p as above, and therefore x* is indeed 
a maximizer of wx over . The number of such choices and hence of loop iterations is 

n (';!'') s fi"- s fi-'-. 

i=l \ '^W j=l j=l 

which is polynomial because A is fixed. In each iteration, we find x{Si, . . . , Sp) maximizing wx over 
or detect SY = by applying the algorithm of Lemma 14.11 using a single query of the linear- 
optimization oracle for S . □ 

We will later show that, for a suitable choice of A , we can guarantee that, for every block of 
the partition of S induced by A , the naive strategy applied to does give a good solution, with only 
a constant number of better objective values obtainable by solutions within 5*^ . For this, we proceed 
next to take a closer look at the monoid generated by a p-tuple a and at suitable restrictions of this 
monoid. 



5 Monoids and Frobenius Numbers 

Recall that a p-tuple a = (oi, . . . , ap) is primitive if the are distinct positive integers having greatest 
common divisor gcd(a) = gcd(ai, . . . , ap) is 1 . For p = 1 , the only primitive a = (ai) is the one with 
oi = 1 . The monoid of o = (ai, . . . , ap) is the set of nonnegative integer combinations of its entries, 

M(a) = {fJ-a = J2^^^ fiiOi : G Z^} . 

The gap set of a is the set G{a) := Z+ \ M(a) and is well known to be finite [4]. If all > 2 , then 
G{a) is nonempty, and its maximum element is known as the Frobenius number of a , and will be 
denoted by F(a) := maxG(a) . If some = 1 , then G{a) = , in which case we define F(a) := by 
convention. Also, we let F(a) := by convention for the empty p-tuple a = {) with p = . 

Example 5.1. If a = (3,5) then the gap set is G{a) = {1,2,4,7} , and the Frobenius number is 
F(a) = 7 . 

Classical results of Schur and Sylvester, respectively, assert that for allp > 2 and all a = (ai, . . . , ap) 
with each Oj > 2 , the Frobenius number obeys the upper bound 

(4) F(a) + 1 < mm{{ai-l){aj -1) : 1 < i < j < p} , 

with equality F(a) + 1 = (ai — l)(a2 — 1) holding for p = 2 . See [4j and references therein for proofs. 
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Define the restriction of M(a) by A G to be the following subset of M(a) : 

M{a, A) := {fia : fi e , ^ < X} . 

We start with a few simple facts. 

Proposition 5.2. For every A G , M(a, A) is symmetric on {0, 1, . . . , Aa} , that is, we have that 
g € M(a, A) i/ and only if Xa — g (z M{a, A) . 

Proof. Indeed, g = fia with < < A if and only if Aa — 5 = (A — /i)a with 0<A — /x<A. □ 

Recall that for z,s ^"L and Z C Z , we let z + sZ := {z + sx : x E Z} . 
Proposition 5.3. For every X G iF^ , we have 

(5) M(a,A) C {0,l,...,Aa}\(G(a)U(Aa-G(a))) . 

Proof. Clearly, M(a, A) C {0, 1, . . . , Aa} \ G{a) . The claim now follows from Proposition 15.21 □ 

Call A G saturated for a if ([5]) holds for A with equality. In particular, if some a^ = 1 , then A 
saturated for a implies M(a, A) = {0, 1, . . . , Aa} . 

Example 15.11, continued. For a = (3, 5) and say A = (3, 4) , we have Aa = 29, and it can be easily 
checked that there are two values, namely 12 = 4 • 3 + • 5 and 17 = 4 • 3 + 1 • 5 , that are not in 
M(a, A) but are in {0, 1, ... , Aa} \ ( G{a) U (Aa — G{a)) ) . Hence, in this case A is not saturated for a . 

Let max(a) := maxjai, . . . , Op] . Call a = (ai, . . . , ap) divisible if a^ divides Oj+i for i = 1, . . .p—1 . 
The following theorem asserts that, for any fixed primitive a , every (component- wise) sufficiently large 
p-tuple A is saturated for a . 

Theorem 5.4. Let a = (ai, . . . , Op) be any primitive p-tuple. Then the following statements hold: 

1. Every X = (Ai, . . . , Ap) satisfying Aj > max(a) for i = 1, . . . ,p is saturated for a . 

2. For divisible a , every X = (Ai, . . . , Ap) satisfying Aj > — 1 for i = 1, . . . ,p — 1 is saturated 
for a . 
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Proof. We begin with Part 1. As we go, we make some claims for which we employ somewhat tedious 
and lengthy elementary arguments to carefully verify. We relegate proofs of these claims, specifically 
Claim 1 and Subclaims 2.1-2.4, to the Appendix. 

Suppose that Aj > max(a) , for i = 1, . . . ,p . Suppose that the result is false. Then there is a p- 
tuple /i G so that fia < Xa but /ia ^ M{a, A) . By Proposition 15.21 we can assume that /io < ^Aa . 
Among all such fj, , choose one that has minimum violation Yl^.=iif^i ~ ^i)^ ■ Let j be an index such 
that fij > Xj . 

Claim 1: There are at least two indices k for which < Aa;/2 . 

Next, for every integer < 7 < — 1 , consider the two- variable integer linear program: 

min xi{^) 

Pj s.t. ajXji-y) - aixii-y) = 70^ ; 

Xjil) , xi{j) G Z+ . 

Claim 2: For some 7 < [aj/2] , there is a nonzero optimal solution to , such that xi{^) < [aj/2j . 

Proof of Claim 2: For the purpose of establishing Claim 2, we assume, without loss of generality, that 
gcd(aj, Ofc, a;) = 1 ; if this did not hold, we could just divide the integers 0^,0^,0/ by their greatest 
common divisor, thus proving a stronger result. 



Subclaim 2.1: The integer program P^ is feasible for all integers < 7 (< — 1) that are integer 
multiples of gcd(a;,aj) . 

Subclaim 2.2: In fact, for 7 = Zfcgcd(a/,aj) with G Z_|- , we have that x^{'^) = zigcd{ak,aj) for 
some zi G . 

Subclaim 2.3: For < 7, 7' < Oj/ gcd(afc, aj) , we have that ^^(7) 7^ Xi{'y') for 7 7^ 7' • 
Subclaim 2.4: For integer 7 > Oj/ gcd(a/c, aj) , we write 7 uniquely as 

7 = 7' + MOj/ gcd(afe,aj) , 

with n G Z+ , 7' G Z+ , 7' < aj/ gcd{ak, aj) . Then we have that 

xiil) = xli'j) , 

Xjil') = 2;* (7) + fiOk/ gcd{ak,aj) . 
Now we are in position to complete the proof of Claim 2. First, if gcd(a/,aj) > 2 , then Claim 2 
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follows because 



x/(0) := aj/ gcd{ai,aj) , 
Xj{0) := ai/ gcd{ai,aj) 

is a feasible solution of Pq with xi{0) < [aj/2\ . So, we can assume from now on that gcd{ai,aj) = 1 . 



We denote by J7 the set of all integers < 7 < — 1 for which is feasible. Next, assume that 
gcd(afc, aj) > 2 . Then by what we have shown already, 

{x;(7) : 7 G 17} = {x;(7) : 7 G , 7 < a^/ gcd(afe, Cj)} . 

Because Uj/ gcd{ak,aj) < aj/2 , there is a 7 < aj/ gcd{ak,aj) < aj/2 such that P^ has a feasible 
solution with xi^j) = 1 . So we now can further assume that gcd(a,fc, aj) = 1 . 

Then X;*(7) ^ x'^i^') for all 7 G , 7 ^ 7' implies that the cardinality of the set {^^(7) : 1 < 7 < 
[aj/2]} is equal to [aj/2] . Because Xi{^) is an integer between and aj — 1 , it follows that there 
must exist a 7* with 1 < 7* < [aj/2] such that x^(7*) < [aj/2j . Hence we have established Claim 2. 

Notice that this then also implies that 

x*{'y*) Oj = 7*aA,. + x'^i^j*) ai < max(a) (7* + 2;;*(7*)) < max(a) aj , 

which implies x*^^*) < max(a) . 
Now, define a new p-tuple v by 

i^j := Hj - x*{j*) , vi:= ni+x*i{'^*) , i^k ■= fJ'k + 7* , and i^i := m for all i^j,k,l. 

Because x*{'y*) < max(a) , it follows that z^j > . Moreover, for i G {k, , < < Aj . Therefore 
I' is nonnegative, satisfies va = = v , and has lesser violation than /i , which is a contradiction to 
the choice of ^ . So indeed v G M(a, A) , and we have established Part 1 of the theorem. 

Before continuing, we note that a much simpler elementary argument can be used to establish Part 
1 of the theorem under the stronger hypothesis: Aj > 2max(a) for i = 1, . . . ,p . 



We next proceed with establishing Part 2 of the theorem. We begin by using induction on p . For 
p = 1 , we have ai = 1 , and every A = (Ai) is saturated because every < f < Aa = Ai satisfies 
V = fia = fii ior fi < X given by /i = (^1) with fii = v . 

Next consider p > 1 . We use induction on Ap . Suppose first that Ap = . Let a' := (ai, . . . , Up-i) 
and A' := (Ai, . . . , Ap_i) . Consider any value < v < Xa = X'a' . Since A' is saturated by induction on 
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p , there exists fi' < X' with v = fi'a' . Then, fi := (/x', 0) < A and w = /ua . So A is also saturated. Next, 
consider Ap > . Let r := (Ai, . . . , Ap_i, Ap — 1) . Consider any value < v < ra = Xa — Up . Since r 
is saturated by induction on Ap , there is a < r < A with v = fxa , and so f G M{a, r) C M(a, A) . 
Moreover, v + ap = fia with /x := (/xi, . . . , fip-i, /Xp + 1) < A , so f + Op G M(a, A) as well. Therefore 

(6) {0,1,..., ra} U {op, Op + 1, . . . , Xa} C M(a,A). 
Now, 

p p-i p-i . p-i 

ra = ^nai > ^ AjOj > f - 1 ) = ^(oj+i - Oj) = ap - 1 , 
1=1 i=l 1=1 ^ ^* ^ i=l 

implying that the left-hand side of (0) is in fact equal to {0,1,..., Aa} . Therefore A is indeed saturated. 
This completes the double induction, the proof of Part 2, and the proof of the theorem. □ 

6 Obtaining an r-Best Solution 

We can now combine all the ingredients developed in the previous sections and provide our algorithm. 
Let a = (oi, . . . , Qp) be a fixed primitive p-tuple. Define A = (Ai, . . . , Ap) by Aj := max(a) for every i . 
For /U < A define 

■= ■ ^^ = ^^> < ■= ( gcd(a. -^EJA) ■ ^^''^) • 

Finally, define 

(7) r{a) := Y.^{a'^^ ' 

The next corollary gives some estimates on r(a) , including a general bound implied by Theorem 15. 4[ 
Corollary 6.1. Let a = (ai, . . . ,ap) be any primitive p-tuple. Then the following hold: 

1. An upper hound on r{a) is given by r(a) < (2max(a))^ . 

2. For divisible a , we have r{a) = . 

3. For p = 2 , that is, for a = (oi, 02) , we have r{a) = F(a) . 
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Proof. Define A = (Ai, . . . , Ap) by Aj := max(a) for every i . First note that if is empty or a singleton 
then is empty or = 1 , and hence F(a^J = . 

Part 1: As noted, F(a^) = for each /i < A with |/^| < 1 . There are at most 2P(max(a))P~^ 
p-tuples /i < A with [/^| > 2 and for each, the bound of equation @ imphes F(a^) < (max(a))^ . 
Hence 

r(a) < 2P(max(a)f"^(max(a))^ < (2max(a)f . 

Part 2: If a is divisible, then the least entry of every nonempty is 1 , and hence F(a^) = for 
every < A . Therefore r(a) = . 

Part 3: As noted, F(a^) = for each fi < X with |/;^| < 1 . For p = 2 , the only /i < A with |/^| =2 
is /U = A . Because = a , we find that r(a) = F(a) . □ 

We are now in position to prove the following refined version of our main theorem (Theorem II. ip . 

Theorem 6.2. For every primitive p-tuple a = (oi, . . . ,ap) , with r{a) as in ^ above, there is an 
algorithm that, given any independence system S C {0, 1}" presented by a linear- optimization oracle, 
weight vector w G {ai, . . . , Cp}" , and function / : Z ^ M presented by a comparison oracle, provides 
an r{a)-best solution to the nonlinear problem min{/(u)x) : x & S} , in time polynomial in n . 
Moreover: 

1. If ai divides Oj+i for i = 1, . . . ,p — 1 , then the algorithm provides an optimal solution. 

2. For p = 2 , that is, for a = (01,02) , the algorithm provide an F{a)-best solution. 

Proof. Consider the following algorithm: 

input independence system S C {0, 1}" presented by a linear-optimization oracle, / : Z ^ M 
presented by a comparison oracle, and w £ {ai, . . . , ap}"' ; 
define A = (Ai, . . . , Ap) by Aj := max(a) for every i ; 
for every choice of p-tuple fi G , < A do 

find by the algorithm of Lemma 14.31 an attaining max{wx : x € S^} any; 

if 5^ 7^ then find by the algorithm of Lemma 13.31 an x* attaining 

mm{f{wx) : x £ {0, 1}" , x < x^} ; 
end 

output X* as one minimizing f{wx) among the x* . 
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First note that the number of p-tuples /i < A and hence of loop iterations and apphcations of the 
polynomial-time algorithms of Lemma 13.31 and Lemma 14.31 is nf=i('^« + 1) = (1 + niax(a))P which is 
constant since a is fixed. Therefore the entire running time of the algorithm is polynomial. 

Consider any p-tuple ^ < A with 5^ 7^ , and let be an optimal solution of m.ax{wx : x G 5^} 
determined by the algorithm. Let I := = {i : jii = Aj} , let g := gcd(aj : i € I) , let 
a := = |(aj : i € /) , and let h := ^{/UjOj : i ^ 1} . For each point x € {0,1}" and for 
each z = 1, . . . ,p , let as usual Aj(x) := |supp(x) n Ni\ , where Ni = {j : Wj = Oj} , and let 
X{x) := {Xi{x) : i G I) . By the definition of Sjl in equation ([3]) and of / above, for each x G we 
have 

wx = '^\i{x)ai + '^Xi{x)ai = ^ /Xjaj + 5 ^ Ai(x)-ai = h + g\{x)a . 

i^I i£l 10 i&I ^ 

In particular, for every x S 5^ we have wx € /i + gM{a) and wx < wx^ = h + g\{x^)a , and therefore 

w-Sl C h + (7(Af(a)n{0,l...,A(x^)a}) . 

Let T := {x : x < x^} . Clearly, for any u < A(x^) there is an x S T obtained by zeroing out 
suitable entries of x^ such that A(x) = P and Ai(x) = Ai(x^) = /Xj for i ^ / , and hence wx = h + gva . 
Therefore 

h + gM{a,X{Xf,)) C w-T . 
Since x^ G , by the definition of 5^ and / , for each i S / we have 

Xi{x^) = |supp(x) n A^'jj > fJ-i = Xi = max(a) > max(a) . 

Therefore, by Theorem 15.41 we conclude that A(x^) = (Aj(x^) : i G /) is saturated for a and hence 

M(a,A(x^)) = (M(a)n{0,l...,A(x^)a}) \ (A(x^)a - G(a)) . 

This implies that 

wSl^\wTCh + 5 (A(x^)a - G(a)) , 

and hence 

\wS;:\wT\< \G{-a)\ = F(a) . 
Therefore, as compared to the objective value of the optimal solution x* of 

mm{f{wx) : x G T} = min{/(wx) : x < x^} 

determined by the algorithm, at most F(a) better objective values are attained by points in . 
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Since S = \S^<x by Proposition 14.21 the independence system S has altogether at most 

better objective values f{wx) attainable than that of the solution x* output by the algorithm. There- 
fore X* is indeed an r(a)-best solution to the nonlinear optimization problem over the (singly) weighted 
independence system. □ 

In fact, as the above proof of Theorem 16.21 shows, our algorithm provides a better, g{a)-hest, 
solution, where g{a) is defined as follows in terms of the cardinalities of the gap sets of the subtuples 
with A defined again by Aj := 2max(a) for all i (in particular, g{a) = \G{a)\ for p = 2), 

(8) 9{a) := Y.\GK)\ ' 

7 Finding an Optimal Solution Requires Exponential Time 

We now demonstrate that our results are best possible in the following sense. Consider a := (2,3). 
Because F(2,3) = 1, Theorem 11.11 (Part 2) assures that our algorithm produces a 1-best solution 
in polynomial time. We next establish a refined version of Theorem 11.2^ showing that a 0-best (i.e., 
optimal) solution cannot be found in polynomial time. 

Theorem 7.1. There is no polynomial time algorithm for computing a 0-best (i.e., optimal) solution 
of the nonlinear optimization problem mm{f(wx) : x E S"} over an independence system presented by 
a linear optimization oracle with f presented by a comparison oracle and weight vector w G {2,3}"'. 
In fact, to solve the nonlinear optimization problem over every independence system S with a ground 
set of n = 4m elements with m >2, at least {j^^i) > 2™ queries of the oracle presenting S are needed. 

Proof. Let n := Am with m > 2, I := {1, . . . , 2m}, J := {2m + 1, . . . , 4m}, and let w := 2 • 1/ + 3 • Ij . 
For E C {1, . . . ,n} and any nonnegative integer k , let (^) be the set of all /c-element subsets of E. 
For i = 0,1,2 , let 

Ti := \x = 1a + 1b ■■ Ae( \ ), Bg( .H C {0, 1}" . 

Let S be the independence system generated by Tq U that is, 

S := {ze {0, 1}" : z<x , for some x G Tq U Ts} . 
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Note that the lu-image of S is 

u; • 5 = {0, . . . , 5m} \ {1, 5m - 1} . 

For every y G Ti , let Sy := S Li {y} . Note that each Sy is an independence system as well, but 
with tu-image 

w ■ Sy = {0, . . . ,5m} \ {1} ; 
that is, the ty-image of each Sy is precisely the u'-image of S augmented by the value 5m — 1 . 
Finally, for each vector c^lT- ^ let 

y(c) := {y G Ti : cy > max{cx : x G 5}} . 

Claim: \Y{c)\ < (^'^J for every c G . 

Proof of Claim: Consider two elements (if any) y,z € Y{c) . Then y = 1a + and z = lu + Iv for 
some A,U £ and B,V e (^'ii)- Suppose, indirectly, that U and B ^V. Pick a e A\U 

and V e V \ B . Consider the following vectors, 

a;0 := y -1^ + 1^ G Tq , 

x'^ := z + la-lv & T2 . 

Now y,z G l^(c) and a;'^, G imply the contradiction 

Ca - Cv = cy - cx^ > , 

Cy — Ca = CZ — CX^ > . 

This implies that all vectors in Y{c) are of the form 1a + 1b with either A G {^^^i) fixed, in which 
case |F(c)| < (^^^J, or B G (^^J fixed, in which case \Y{c)\ < Q"^^) = ) , as claimed. 

Continuing with the proof of our theorem, consider any algorithm, and let , . . . , G be the 
sequence of oracle queries made by the algorithm. Suppose that p < (^^^) • Then 



p 



•i=i 



2m 
m — 1 



This implies that there exists some y G Ti that is an element of none of the y(c*) , that is, satisfies 
c*y < max{c*.T : x G S} for each i = l,...,p . Therefore, whether the linear optimization oracle 
presents S 01 Sy , on each query c' it can reply with some G attaining 

c*x* = max{c*x : x & S} = max{c*a; : x G Sy} . 

Therefore, the algorithm cannot tell whether the oracle presents S or Sy and hence can neither compute 
the w-im&ge of the independence system nor solve the nonlinear optimization problem correctly. □ 
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8 Discussion 



We view this article as a first step in understanding the complexity of the general nonlinear optimiza- 
tion problem over an independence system presented by an oracle. Our work raises many intriguing 
questions including the following. Can the saturated A for a be better understood or even character- 
ized? Can a saturated A smaller than that with Aj = max(a) be determined for every a and be used 
to obtain better running-time guarantee for the algorithm of Theorem 11.11 and better approximation 
quality r(a) ? Can tighter bounds on r(a) in equation ([7]) and g{a) in equation ([8]) and possibly formu- 
las for r(a) and g{a) for small values of p, in particular p = 3, be derived? For which primitive p-tuples 
a can an exact solution to the nonlinear optimization problem over a (singly) weighted independence 
system be obtained in polynomial time, at least for small p, in particular p = 2 ? For p = 2 we know 
that we can when oi divides a2 , and we cannot when a := (2, 3) , but we do not have a complete 
characterization. How about d = 2 1 While this includes the notorious exact matching problem as a 
special case, it may still be that a polynomial-time solution is possible. And how about larger, but 
fixed, d ? 

In another direction, it can be interesting to consider the problem for functions / with some 
structure that helps to localize minima. For instance, if / : M ^ M is concave or even more generally 
quasiconcave (that is, its "upper level sets" {z G M : f{z) > /} are convex subsets of M , for all / G M ; 
see [1], for example), then the optimal value mm{ f{wx) : x G S*} is always attained on the boundary 
of conv(t<; ■ S) , i.e., if x* is a minimizer, then either wx* = or wx* attains max{wx : x G S"} , so 
the problem is easily solvable by a single query to the linear-optimization oracle presenting S and a 
single query to the comparison oracle of / . Also, if / is convex or even more generally quasiconvex 
(that is, its "lower level sets" {z G M : f{z) < /} are convex subsets of M , for all / G M), then a 
much simplified version of the algorithm (from the proof of Theorem 16. '2p gives an r-best solution as 
well, as follows. 

Proposition 8.1. For every primitive p-tuple a = (oi, . . . , Op) , there is an algorithm that, given inde- 
pendence system S C {0, l}" presented by a linear- optimization oracle, weight vector w G {ai, . . . , Cp}" , 
and quasiconvex function / : M — > M presented by a comparison oracle, provides a (max(a) — l)-best 
solution to the nonlinear problem mm{ f{wx) : x (z S} , in time polynomial in n . 

Proof. We could describe the construction as a specialization of the algorithm from the proof of 
Theorem 16.21 but it is more clear to just present it directly. We first use our linear-optimization oracle 
to find X* attaining maxjwx : x G 5} . Then, by repeatedly, and in an arbitrary order, decreasing a 
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single component of the point by unity, we obtain a sequence of points 

x'' ■= X* > x^-^ > ... > x° := , 

with k = X^j=i Xj < n . Let / := min{f{'wx^) : < t < k} . 

Next, using the comparison oracle (a linear number of times), we find the least and greatest indices 
t , say tmin aud tmax respectively, for which x* minimizes f{wx^) . Quasiconvexity of / implies that 

f{wx^) = / , for tmin < t < traa.x ■ 

Moreover, quasiconvexity implies that there is an index s , satisfying tmin — 1 < s < tmax , such that 
all points z € [0,ii;x*] Pi Z having f(z) < f are in [wx^ + l,wx^~^^ — 1] H Z (that is, in one of the 
imax — ^min + 2 intervals [wx^ ,wx^~^^] beginning with the one immediately to the left of tmin and ending 
with the one immediately to the right of tmax — and not the endpoints of that interval). 

The result now follows by noticing that 

wx^~^^ — wx^ < max(a) , for t = 0, . . . , fc — 1 , 
in particular for t = s . □ 

In yet another direction, it would be interesting to consider other (weaker or stronger) oracle pre- 
sentations of the independence system S. While a membership oracle suffices for nonlinear optimization 
when 5" is a matroid [2j, in general it is much too weak, as the following proposition shows. 

Proposition 8.2. There is no polynomial time algorithm for solving the nonlinear optimization prob- 
lem mm{f{wx) : X € S"} over an independence system presented by a membership oracle with f pre- 
sented by a comparison oracle, even with all weights equal to 1, that is, forp = 1, a = 1, u; = (1, . . . , 1). 

Proof. Let n := 2m , let w := ^27=1 1« = (1) • • • 1 1) ) and let 

S := {x £ {0, 1}" : supp(x) < m - 1} . 

For each y E {0, l}*^ with supp(y) = m , let Sy := S U {y} . Note that 

w • S = {0,1, . . . ,m — 1} , w ■ Sy = {0,1, ... ,m — l,m} . 

Now, suppose an algorithm queries the membership oracle less than (^) times. Then some y G {0, 1}" 
with supp(y) = m is not queried, and so the algorithm cannot tell whether the oracle presents S or Sy 
and hence can neither compute the image nor solve the nonlinear optimization problem correctly. □ 



18 



Acknowledgment 



This research was supported by the Mathematisches Forschungsinstitut Oberwolfach during a stay 
within the Research in Pairs Programme. 

References 

[1] Avriel, M., Diewert, W.E., Schaible, S., Zang, I.: Generahzed concavity. Mathematical Concepts 
and Methods in Science and Engineering, 36. Plenum Press, New York (1988) 

[2] Berstein, Y., Lee, J., Maruri-Aguilar, H., Onn, S., Riccomagno, E., Weismantel, R., Wynn, H.: 
Nonlinear matroid optimization and experimental design. SIAM Journal on Discrete Mathematics 
(to appear) 

[3] Berstein, Y., Onn, S.: Nonlinear bipartite matching. Discrete Optimization 5:53-65 (2008) 

[4] Brauer, A.: On a problem of partitions. American Journal of Mathematics 64:299-312 (1942) 

[5] Mulmuley, K., Vazirani, U.V., Vazirani, V.V.: Matching is as easy as matrix inversion. Combina- 
torica 7:105-113 (1987) 

[6] Papadimitriou, C.H., Yanakakis, M.: The complexity of restricted spanning tree problems. Journal 
of the Association for Computing Machinery 29:285-309 (1982) 

Appendix 

Claim 1: There are at least two indices k for which < ^k/"^ ■ 
Proof of Claim 1: We note that < Xj trivially implies 



(9) 



< aj (nj — Xj — 1) . 



Also, fia < ^Xa can be written as 



(10) 




Now, adding ([9]) and ([TO]) , we obtain 



(11) 
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The right-hand side of (jlip is negative, therefore the left-hand side must also be negative. Suppose 
that there is but a single index k for which a summand on the left-hand side of (jlip is negative. Then, 
we have 

Ofe ink - Aa:/2) < -aj (Aj/2 + 1) , 

which implies 

(12) max(a) (/u^ — Afc/2) < —aj (max(a)/2 + 1) . 

We observe that we must have fik — > —aj , otherwise we could decrease the violation by 
decreasing fij by and increasing by aj . But fik — > — implies that 

(13) /ifc - Afc/2 > -aj + 1 + Afc/2 > -Oj + 1 + max(a)/2 . 

Next, we combine (I12p and (I13p to arrive at 

max(a) {—aj + 1 + max(a)/2) < —aj (max(a)/2 + 1) , 

or, equivalently, 

Oj (max(a)/2 — 1 ) > max(a) (max(a)/2 + 1) , 

which cannot hold. 

So Claim 1 is established. 



Subclaim 2.1: The integer program is feasible for all integers < 7 (< Oj — 1) that are integer 



multiples of gcd(a/,aj) . 

Proof of Subclaim 2.1: Suppose that j := Zk gcd{ai, aj) , for some Zk € Z+ . 
By Bezout's Lemma, there are integers Pj , f3i such that 

ajf3j + aiPi = gcd(a/, aj) . 

Moreover, there is an infinite family indicated by 

Oj (^Pj + tail gcd(a/, a^)^ + ai (^i - toj/ gcd(az, a^)^ = gcd(a/, aj) , 

with t ranging over Z . 

Multiplying through by ZfcOfc , and rearranging terms, we obtain 



Oj ( Zkttk ( I3j + tail gcd(az, aj) ) ) + a; ( z^Ofc ( A - taj/ gcd(a/, aj) ) ) = Zk gcd{ai,aj)ak 

= lak ■ 
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Now, for a sufficiently large positive integer t , we will have 



(3i - taj/ gcd{ai,aj) < , 

and so 



Xjh) ■■= Zkak(^Pj +tai/ gcd{ai,aj)^ ; 
-xi{j) := Zkak(^Pi - taj/ gcd{ai,aj) 



will be a feasible solution to . Thus we have established SubClaim 2.1. 



Subclaim 2.2: In fact, for 7 = Zfc gcd(a/, Oj) with z^. G Z+ , we have that X;*(7) = 2; gcd(aA:, Oj) for 
some zi € Z+ . 

Proof of Subclaim 2.2: 

ajX*{'y)/ gcd{ak,aj) - 70^/ gcd(afc, a^) j gcd(afc, Oj) . 
As gcd(aA;, Uj) divides both aj and Ok , we have 

aixUl) = z gcd{ak,aj) , 

for some z G , and hence 

xKl) = {z/ai)gcd{ak,aj) . 

As gcd(a;, gcd(afc, aj)) = 1 , it is clear that ai must divide z (after all x*{'y) G Z), and hence SubClaim 
2.2 is established. 

Subclaim 2.3: For < 7,7' < aj/ gcd(afc, aj) , we have that X;*(7) 7^ a^;*(7') for 7 7^ 7' • 

Proof of Subclaim 2.3: Suppose the contrary. Without loss of generality, 7' > 7 . Then we have the 
following two equations: 

(14) ak-f' + aix*i{-i') = x*{^')aj ; 

(15) + aix*i{-f) = x*{'y)aj . 



Subtracting (115]) from ()T4l) gives 

(16) akil'-l) = a,{x*{Y)-x*{j)) 
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Because 7' > 7 , the left-hand side of (fT6]) is positive, which imphes that x*{'y') — x*{^) > . 

But 7' — 7 < aj/ gcd{ak,aj) . This contradicts that gcd(aj/ gcd(afc, Cj), a^/ gcd(afc, aj)) = 1 , 
because every positive integer solution of a^Xk = ajxj is a positive multiple of 

Xk ■■= aj/ gcd(afc,aj) , 
Xj := flfc/ gcd(afc,aj)) . 

Thus we have established Subclaim 2.3. 

Subclaim 2.4: For integer 7 > aj/ gcd(afc, aj) , we write 7 uniquely as 

7 = 7' + MOj/ gcd(afc,aj) , 
with fi € Z+ , 7' G Z+ , 7' < aj/ gcd(afc, Oj) . Then we have that 

x^ij') = xti7), 

Xjil') = 2;* (7) + iiak/ gcd{ak,aj) . 

Proof of Subclaim 2.4-' We can directly check feasibility: 

(^j W + ^«fc/gcd(afe,aj)) + a/X;*(7) = (7' + fxaj / gcd{ak, aj)) ak . 

Moreover, there is no feasible solution x for Py having x/ < 2:^(7) , because if there were, we would 
simply add /ucfc/ gcd(afc, aj) to Xj , and leave x/ unchanged, to produce a feasible solution for P^ having 
objective value less than x^{'y) , a contradiction. Thus we have established Subclaim 2.4. 

Jon Lee 

IBM T.J. Watson Research Center, Yorktown Heights, NY 10598, USA 
email: jonleeQus. ibm. com, http://www. research, ibm. com/people/ j/jonlee 

Shmuel Onn 

Technion - Israel Institute of Technology, 32000 Haifa, Israel 
email: onn@ie.technion.ac.il, http://ie.technion.ac. il/^onn 

Robert Weismantel 

Otto-von-Cuericke Universitdt Magdeburg, D-39106 Magdeburg, Cermany 

email: weismantel@imo.math.uni-magdeburg.de, http://www.math.uni-magdeburg.de/^weismant 



22 



